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Abstract 


Deblurring capabilities would significantly improve the scientific return 
from Space Shuttle Crew acquired images of the Earth and the safety of Space 
Shuttle missions. This summer, deblurring techniques were developed and 
demonstrated on two digitized images that had been blurred in different 
ways. The first was blurred by a gaussian blurring function analogous to that 
caused by atmospheric turbulence while the second was blurred by improper 
focussing. It was demonstrated in both cases that the nature of the blurring 
i.e. gaussian and Airy, and the appropriate parameters could be obtained from 
the Fourier transformation of their images. The difficulties posed by the 
presence of noise necessitated special consideration. It was demonstrated that 
a modified Wiener frequency filter judiciously constructed to avoid over 
emphasis of frequency regions dominated by noise resulted in substantially 
improved images. Even though the deblurred images were similar to the 
original unblurred images, their Fourier transformed images were similar 
not as similar, indicating that more refined techniques applied to the Fourier 
image could result in further improved images. Several important areas of 
future research were identified. Two areas of particular promise are the 
extraction of blurring information directly from the spatial images and 
improved noise abatement from investigations of select spatial regions and 
the elimination of spike noise. 
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Introduction 


The Space Shuttle Earth Observation Office serves as the repository of over 
100,000 photographs form the Manned Space Flight Program. Photographs 
are taken to monitor both the lift-off and the landing of the Space Shuttle, 
and to document environmental, geological, meteorological and 
oceanographic phenomena observed from orbit. Inadvertently a number of 
these are blurred by improper focussing, relative camera-object motion, 
atmospheric distortions i.e. shock waves or thermal gradients, etc. 
Atmospheric distortions are particularly annoying since they restrict NASA's 
ability to monitor the effects of lift-off on the shuttle and the decent of the 
orbiter's wheels during landing. To gain access to important engineering and 
scientific information contained in blurred photographs, it is necessary to 
apply image processing techniques. The more sophisticated of these involve 
digitization of the raw image and subsequent computer processing of the 
degitized images or their Fourier transforms. 

The research accomplished as a Summer Faculty Fellow involved the 
development and implementation of a deblurring procedure to handle a) the 
digital gaussian blurring which is analogous to that caused by atmospheric 
turbulence and b) blurring due to improper focussing e.g. actual focal lengths 
differing from expected values due to thermal effects, etc. Although there is 
no apparent indication of the source of blurring in the spatial images, there 
are definite indications of the source of blurring in the Fourier (transformed) 
images. Thus to improve a spatial image one first diagnoses the Fourier 
images for signs of the source of blurring and then used that information to 
construct a frequency "filter" that reduces the effects of blurring in the Fourier 
image which in turn would produce a sharper spatial image. However, the 
presence of unknown noise in the original images and increased by the 
digitalization process, necessitates special consideration. In particular, even 
though blurring reduces the high frequency components compared to the 
lower frequency ones, simple enhancement of the higher frequency 
amplitudes will not necessarily improve the image, since it could increase the 
overall noise to signal ratio by introducing high frequency amplitudes that 
are noise dominated. The trick is to judiciously emphasize only those 
frequency regions where the signal to noise ratio is acceptable. There are 
many excellent text dealing with digital processing (1,2,3) and discrete Fourier 
transforms (4,5) that provide the background for the work described in this 
report. 
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II Theory and Approach 


II.l Blurring as a Convolution Process: the Point Spread Function 

In general a blurred image is the result of light that would have arrived at 
a particular point being spread over a number of points. If the distribution is 
the same for all points in the image and if the light is incoherent then in the 
absence of noise the final intensity at a point (pixel) m,n would be given by 
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where P.S.F. is the point spread function, i.e. the image of an ideal point 
source in the object plane, f s m<n is the sharp image that would have formed 
in the absence of blurring and noise. 

For the images of interest the digitization process yields an image made up 
of NxN pixels (N=512). Due to the blurring bringing light rays into the image 
region that would have otherwise not nave entered it, the summation in this 
expression should include m"s and n"s outside the range of 0 to N-l. 
Nevertheless, the sum is usually assumed to be limited to this range to avoid 
having to deal with an "underdetermined problem" (6). 

Noise is always present in the image. It can be the result of electronic 
cross-talk, round-off errors in the digitization process, random, 
inhomogeneous chemical processes in the original film, etc. Noise can be 
divided into two contributions: one independent and the other dependent on 
the signal, i.e. f s (7). In the latter case it could be incorporated into the point 
spread function. Since we have little knowledge of the nature of the noise, 
except that it is usually dominant at high frequencies, we will describe it by an 
unknown function n m , n and write the expression for the blurred image as 
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The next step is to invert the sum in equation 1 and solve for f s . Since 
the summation is usually referred to as convolution, the inverse process is 
referred to as deconvolution. The basic technique is to expand all functions 
in terms of a complete set of basis functions which will turn the summation 
into a simple product of the expansion coefficients. 
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11,2 Fourier Transformations 


For simplicity we use the traditional discrete Fourier expansion 

f m/ n = ^2 Fk. l exp 2 7c i( km +ln)/N (2a) 

k, 1 

where there are N values of k and 1. 

The inverse Fourier transformation is then given by 

F k , l = (1 /N 2 ) ^ n ex P ” 2 K ^ km + ln ^ / N (2b) 

k, 1 

In the following we will use lower case letters , e.g. f, for the spatial images 
and upper case letters, e.g. F, for the Fourier transformed image. The Fourier 
transform of the point function,P.S.F., is called the modulation transfer 
function,M.T.F.. 

In terms of the Fourier transformed variables equation 1 takes the form 
F b k , i = F s k/1 • M.T.F k/ j + N k> i (3a) 

Deleting the common subscripts and solving this equation for the Fourier 
transform of the sharp image function yields 

ps = Fb/M.T.F. - N /M.T.F. (3b) 


n.3 Noise Considerations 


If the modulation transfer function and the noise, N, were both known 
then equation 3b could be used to find F s and a subsequent inverse Fourier 
transformation would yield a sharp image. For the sake of discussion let us 
assume that we have a reasonably good understanding of the blurring process 
and have a good approximation to P.S.F. and thus to M.T.F. Most likely the 
blurring extends over a few spatial pixels and M.T.F. will be small at high 
frequencies, i.e. large values of k and 1. Blurring due to improper focussing, 
relative motion, atmospheric turbulence are in general of this type. The first 
two sources of blurring have M.T.F.'s that vanish on contours in the k,l space. 
Where ever M.T.F. is small the contribution from noise, N/M.T.F., in 
equation 3b can be significant. Consequently, approximating F 5 by 
F b /M.T.F.can result in large noise contributions. The normal procedure to 
avoid this problem is to multiply F b by a "filter" function which uses 
information about the noise to approximate 1 /M.T.F. where noise is 
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unimportant and avoids contributions from regions where noise dominates. 
The Wiener filter is the most common filter discussed in the literature 
(23,4,6): 

W-l = M.T.F. •( 1 + I N/FS-M.T.F. 1 2 ) 

As it stands this expression requires knowledge of the amplitude of the 
function to be calculated and the unknown noise contribution. A simple way 
to suppress noise is to replace N/F s with a constant (3) or a constant times a 
gaussian, e.g. C exp c(k2+l2). in the former case the constant could be chosen 
to be the value of M.T.F. at frequency in the range where the maximum value 
of W is desired. A gaussian would do a better job of modelling N/Fs than a 
constant but it involves finding appropriate values for two instead of one 
parameter. 

Another approach is to notice that when M.T.F. vanishes F b is equal to N. 
Consequently, if F b o is the value of F b where M.T.F. = 0, then (F b - 
F^ 0 ) /M.T.F. would avoid the introduction of excessive noise, since - F b o 
removed the noise. 


Ill Applications : A Tale of Two Images 


In this section we discuss two applications of the deblurring techniques 
described in the previous section. The first application involves an image 
blurred digitally by convolution with a gaussian spread function. This is the 
digital analogy of atmospheric turbulence for long exposure times(l,8). The 
second involves an image blurred by defocussing the lens of the digitizer and 
thus is an example of blurring due to improper focussing. 

nil Enhancement of an Image Blurred by Gaussian Convolution 

Figure la shows the original image and figure lb shows the image blurred 
by convolution with a point spread function which is gaussian within a 5 by 5 
square of pixels. Figure lc shows an enhanced image obtained by applying a 
filter to the Fourier transform of the blurred image. It should not be 
considered an ultimate product but simply a demonstration of the results that 
could be obtained without expenditure of the time needed to fine tune the 
enhancement process. Figures 2a,b and c show the Fourier transformed 
images corresponding to those shown in figure 1. Clearly the Fourier image 
(figure 2c) reconstructed from the Fourier transform of the blurred image 
(figure 2b) is an improvement but still differs considerably from the Fourier 
transformation of the sharp image (figure 2a) especially in the intermediate to 


6-6 



high frequency region. Figure 3 shows the inverse of the deblurring filter 
used to obtain figure 2c from figure 2b. 

There were three important aspects in constructing the deblurring, filter: 

1. Only in a 5 by 5 spatial pixel region was the P.S.F. non-zero. 

2. Inside that region the P.S.F. could be described by a single coefficient 
in the exponent of the gaussian. 

3. The filter would have to deviate sufficiently form 1/M.T.F. in the 
high frequency region where 1/M.T.F. would be large and noise 
could be expected to dominate. 

The Fourier images shown in figures 2a and 2b were used to obtain the 
coefficient for the "gaussian" M.T.F. This was done by taking the Fourier 
transform of their ratios which in the absence of noise would be the P.S.F. 
Although the resulting function did not vanish outside the 5 by 5 pixel 
region, presumably due to noise, there was sufficient accuracy within that 
region to determine the coefficient within a 5% variance. 

Knowing that in the continuous case the Fourier transform of a gaussian 
is again a gaussian, a preliminary filter was constructed using a gaussian 
M.T.F. The resulting image was unrecognizable, presumably due to high 
frequency noise. Next a one constant Wiener-like filter (see section II.3) 
which emphasized the medium to low frequencies (N/ V6) was used. This 
gave a reasonable but rather grainy image. The final result shown in figures 
1 c and 2c was obtained by replacing the gaussian M.T.F. in the Wiener-like 
filter by the M.T.F. resulting from a P.S.F. having only a gaussian distribution 
within a 5 by 5 pixel region. The squareness of this pixel region gave extra 
emphasis to the high frequency components near the axes and consequently 
removed most of the graininess. Further refinements could be expected from 
a more judicious choice of the constant in the filter or optimization of a two 
constant filter (see section II.3). 

m.2 Enhancement of an Image Blurred by Improper Focussing 

Figures 4a, b and c show the original, blurred and the first order restored 
image and figures 5a, b and c show their respective Fourier transforms. The 
first two images were made using the video digitization system. Though it is 
no evident from the figures as shown, the top 1/16 of both images is missing 
as a result of the digitization process. Such spatial-domain truncation causes 
over shooting or ringing in subsequent filtered image (p. 26 ref. 1). A crude 
attempt was made to fill in these region, which replaced a few strong 1—0 
constant frequency by the scratchy like region of with N/16 centered on the 1 
axis as shown in figure 5b. It is thought that remnants of this spatial domain 
truncation are the cause of the ringing seen in figure 4c. 

Knowing that the blurring process is due to defocussing one can expect to 
see the zero contours of the appropriate M.T.F. in the Fourier transform of 
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the blurred image. If the P.S.F. is a uniform disk, the the M.T.F. is the 
discrete equivalent of the Airy function. A subroutine was then written to 
generate Airy-like M.T.F. 's for a uniform solid disk of variable radius 
surrounded by an annulus whose value decreases from that of the disk to 
zero at a variable outer radius. The halo visible where F^ drops down to the 
background noise, was used to estimate the radius of the disk. Then using 
this estimate, Airy-like M.T.F.'s were generated. A disk with a diameter of 
about five pixels and a fuzzy annulus of two pixels width was selected. 
Comparing it to the F^ shown in figure 5b, it was seen that the first zero 
contour was in the halo region and that the second zero contour just enclosed 
the region where F^ was still significant. Thus the Fourier transform of an 
image blurred by improper focussing contained sufficient information to 
identify by visual inspection an appropriate M.T.F. However, there were 
other M.T.F.'s resulting from varying the disk radius and the annulus width 
that had similar zero contours. The one chosen gave slightly better results 
although the fuzzy annulus had little of no effect on the ringing. Having an 
appropriate M.T.F. , a Wiener-like filter was chosen that gave maximum 
emphasis in a frequency circle of radius 80 pixels. Clearly the resulting. 
Fourier image represents an improvement but is not identical to that of the 
unblurred image. Perhaps the most improvements in the deblurred image 
are to the letters on the wings of the orbiter and the upper regions of the solid 
fuel rockets. 


IV Future Research 


The research described herein revealed several interesting avenues of 
future work. Two prominent areas of importance in designing the deblurring 
filters are the identification of the appropriate M.T.F. from Fourier transforms 
of the blurred images and noise abatement. The latter area of work may lead 
to preprocessing the blurred images, e.g. with windowing functions. Another 
important investment of research time is that of progressing systematically 
through a series of blurred images. These images should be classified as 
much as possible beforehand into groups having similar blurring processes. 

It is clear that any strategies developed in working with a few images need to 
be tested and refined by considering other images. In this section each of 
these areas of work will be reviewed. 
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V.l Construction of M.T.F.'s from Images 


Each blurring process can be expected to have its own point spread 
function (P.S.F.) and an associated modulation transfer function (M.T.F.). A 
priori knowledge of the blurring process is helpful, but not essential. For 
example, the circular halo observed in the second but not the first image 
considered in section III is characteristic of improper focussing. The lack of a 
halo would suggest another type of blurring, e.g. gaussian. Relative motion 
would have a line segment for a P.S.F. For the simplest case, that of a straight 
line segment, i.e. constant velocity, the resulting M.T.F. would have 
rectangular symmetry. A combination of improper focussing and relative 
motion has a M.T.F. that is the product of the individual M.T.F.'s for each 
process. Blurring due to atmospheric turbulence under certain conditions 
would be gaussian and would not result in a halo pattern. In many cases, the 
only means of identifying the M.T.F. and the parameters needed to describe 
it will be from the Fourier image. Visual inspection of a Fourier image for 
the halo pattern probably is the fastest means of identifying the source of 
blurring. Having appropriate M.T.F. parameters, e.g. the exponent coefficient 
of a gaussian or the diameter of the disk in the case of improper focussing. 
How much of this work could be accomplished by a computer remains to be 
seen. 

In principle, it is possible to extract P.S.F. parameters directly from features 
of the spatial image. In particular, if the image pixels associated with a single 
point of the sharp image can be identified then one has the P.S.F. In theory 
such information is also contained in the profiles of edges and lines in the 
blurred image. The change of pixel values perpendicular to an edge can be 
shown to be proportional to the pixel values of the P.S.F. measured along the 
diameter in the same directions. 

IV.2 Noise Abatement 


The two examples considered in section III have clearly shown that 
identification of the M.T.F. is only the first step. Care must be taken to avoid 
emphasis of noise. The approach used there of a one or two parameter 
Wiener-like suppression of high frequency contributions was a rather crude 
method of noise abatement. It would be better if the source or the nature of 
the noise could be identified and removed by a separate frequency filter. For 
example, in working with the first image, it was found that setting all 
amplitudes inside circular arcs of radius N/4 around the four comers of the 
Fourier image had no visual effect on the unblurred image. In the case of 
spike noise, where individual pixels are radically different in value from the 
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surrounding pixels, a subroutine could search and remove such "sparkle". 
Sparkle is most prominent near the corners of the Fourier image where little 
if any signal is present. It is in contrast to the more cloud-like structure • 
typical of the low frequency region where signal dominates. It should be 
possible to obtain information on noise by studying select portions of the 
spatial image, e.g. if a region is known to be uniform in intensity, then any 
ripple in its Fourier transform may be assumed to be noise. Identifying 
regions or individual pixels of the Fourier image as noise dominated and 
their removal would greatly reduce the importance of the choice of 
parameters in the deblurring filter. 

IV.3 Image Truncation and Windowing 

In dealing with the image blurred by improper focussing, ringing 
supposedly due to spatial truncation was observed. In this particular case, the 
upper sixteenth of the image was not present due to the digitalization process. 
Because of the cut-off of the camera, it is barely noticeable in figures 2a, b or c. 
However, an attempt to fill in the missing area removed strong vertical lines 
near 1=0 and created the scratchy band about 32 pixels wide, i.e. N/16. This is 
an indication of the importance of spatial truncation on the Fourier image. It 
would be worthwhile to see what effect the edges of an image have on its 
Fourier image. There are many different types of window functions(l) that 
remove the effects of the edges and could help us to see which Fourier 
features are important and which are spurious. 

V Summary and Conclusion 

The two examples discussed in section II, demonstrated the significant 
improvement that can be accomplished in enhancing blurred images. 
Information needed to construct the frequency filters which manipulate the 
Fourier images can be extracted from the Fourier as well as the spatial image. 
For the two examples considered, the nature of the blurring mechanism was 
evident from the presence or absence of halos in their Fourier transforms. It 
was also found that once the type of blurring was recognized if was easier to 
use the Fourier images to fix the appropriate parameters. Supposedly 
methods could be developed to extract the needed information from features 
of the spatial images that might complement the work with Fourier images. 
Similarly, noise reduction could proceed from identification of spike noise 
etc. in the Fourier image or from studying select regions of the spatial image. 

In conclusion, it has been demonstrated that blurred images can be 
enhanced using information extracted from the images themselves. 

However, there is plenty of research and art work to be done before JSC has a 
cookbook for deblurring images. 
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Figures la, lb and lc Figures 2a, 2b and 2c 
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Figures 5a, 5b and 5c 
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